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ABSTRACT 



An apparatus and method of template fitting for fitting a 
plurality of templates in correspondence with a gray -scale 
image, including receiving gray-scale image information of 
an observed image; defining a window of the gray-scale 
image information so as to enable a loose -gray-scale fit 
between the defined window and the plurality of templates; 
extracting a plurality of pixels of the image information and 
using the window to which to fit at least one of the plurality 
of templates. Further, fitting one of the plurality of templates 
to the defined window within a threshold looseness interval 
associated with the template wherein a looseness interval 
between the window of image data and the template is less 
than the threshold looseness interval. 

19 Claims, 12 Drawing Sheets 
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BACKGROUND OF THE INVENTION 

1. Field of Invention 

This invention relates to systems and methods for pro- 
cessing images using filters. More specifically, this invention 
relates to systems and methods for designing and imple- 
menting image processing filters using templates wherein 
the filters operate on gray-scale images and the templates 
identify gray-scale image features for the purposes of modi- 
fication or for extracting some image statistic, and for the 
purposes of optimization for the human visual system, or 
compatibility with other system modules, such as, compres- 
sion algorithms, recognition algorithms and those occurring 
in printing and display devices. 

2. Description of Related Art 

A wide variety of digital document processing tasks are 
performed using template-based filters. Illustratively, digital 
document processing tasks include resolution conversion, 
enhancement, restoration, appearance tuning and 
de-screening of images. These tasks are commonly per- 
formed on monochrome and color images, as well as binary 
and continuous tone images. Although, due to the binary 
nature of conventional templates, implementing many digi- 
tal document-processing tasks on continuous tone images 
has been problematic prior to the present invention. A 
continuous tone image may also be referred to as a gray- 
scale image. 

In conventional systems and methods, a typical filter 
includes template operators to perform filtering of the 
images, where, a filter may be characterized as an operator 
or device that transforms one image into another image or 
transforms an image to a collection of information, such as 
image statistics. The filter is formed of a number of imaging 
template operators, often simply referred to as templates. 
These templates may be, for example, stored in a look-up 
table and implemented using a look-up table formalism. Or 
other equivalent formalisms, such as Boolean logic may be 
employed. The number of templates in a filter may vary 
between a small number of templates to thousands of 
templates. Due to its versatility in design, a look-up table is 
typically used to implement a template-based filter. 

A raster is a one-dimensional array of image data, reflect- 
ing a single line of data across a single dimension, i.e., the 
length or the width, of the image. Further, each location, or 
"picture element," in an image may be called a "pixel." In 
an array defining an image in winch each item of data 
provides a value, each value indicating the properties of a 
location may be called a pixel value. Each pixel value is a 
bit in a binary form of an image, a gray-scale value in a 
gray-scale form of an image, or a set of color-spaced 
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coordinates in a color coordinate form of an image. The 
binary form, gray-scale form, and color coordinate form are 
each arranged typically in a two-dimensional array, which 
defines an image. An N-dimensional array is typically used 
for an N-dimensional images, where for example, N=3 for $ 
3 -dimensional topographic images. 

Using the typical binary image processing setting as an 
example, the filter, using the templates, transforms certain 
observed pixel patterns in a binary image, for example, into 
a corresponding enhanced binary pixel pattern. Specifically, 1Q 
the filter observes an arrangement of pixels using a suitable 
window or mask. A window is an imaging algorithmic 
device that observes a plurality of pixels at the same time, 
where the plurality of pixels is located about a target pixel. 
The values and locations of the observed pixels are inputted 5 
into the template matching operations. After observing the 
arrangement of pixels, about a target pixel, the filter then 
attempts to match the observed pixel pattern with one or 
more of the templates in the look-up table. If the look-up 
table contains a match to the observed pixel pattern, the 2Q 
look-up table generates an appropriate output. The output 
may be an enhanced pixel pattern for the target pixel that 
corresponds to the observed pixel pattern. The output could 
also be information in other forms; for example, the output 
could be a code denoting the match condition, or a data to 25 
be used for a statistical characterization of image regions. 

A wide variety of types and sizes of observation windows 
or masks are known. The particular window used in a 
particular application depends on the image to be analyzed 
and the particular process to be performed on the image. 30 
Illustratively, a 3x3 window may be used to process an 
image. The 3x3 window, at various locations in the image, 
observes a 3x3 block, i.e., a 9-pixel block, of binary-valued 
pixels, for example. One pixel in the window is the target 
pixel, which is typically the center pixel, while the other 35 
pixels in the window are the neighboring pixels. Hie target 
pixel and the neighboring pixels form a neighborhood. The 
window is typically scanned across an image advancing 
from target pixel to target pixel. 

After the neighborhood is observed in the window, the 40 
neighborhood is then processed in some manner. For 
example, the observed neighborhood may be transformed 
into a vector The vector is expressed in the form of (x l7 
x 2 . . . Xy) where N As the number of pixels in the 
neighborhood and is used to represent the properties of the 45 
target pixel, including the neighborhood of the target pixel. 
Each element of the vector represents one of the pixels 
observed in the window. The vector is then used in the 
look-up table to generate a desired output, for example. 

A look-up table may be created in a wide variety of ways. 50 
Typically, an input value is input into the look-up table and, 
in response, the look-up table outputs an output value. 
Further, the look-up table is typically created using a training 
image or a set of training images. "Restoration and Enhance- 
ment of Digital Documents," by R. Loce and E. Dougherty, 55 
teaches a variety of methods for designing templates based 
on sets of training images. The training images will occur in 
pairs, where one member is the "typically input image,' 7 or 
the "typically observed image," i.e., the "observed image," 
and the other image is the "ideal desired processed version to 
of the image," i.e., the "ideal image." The training image 
pairs may be input into a computer program that acquires 
and analyzes pattern statistics between the two images, i.e., 
using computer-aided filter design techniques. 

Conventional computer-aided filter design may be accom- 65 
plished through using training-sets of document bitmaps, for 
example. 
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Illustratively, for designing a filter that enhances from a 
binary state to a gray-scale state, for a given pattern that 
occurs in the binary image about a target pixel, a training 
analysis system examines a target pixel at that correspond- 
ing location in the gray-scale image. The center of the 
window may be placed at the target pixel, for example. 
Based on the set of gray-scale pixels in the gray-scale image 
that are associated with corresponding target pixels in the 
binary image and gray-scale image, and associated with a 
similar neighborhood pixel pattern, a "best gray-scale pixel 
value" is determined for processing a target pixel that 
possess that pattern of neighborhood pixels. In other words, 
a template is created for the target pixels in the binary image 
possessing similar neighborhood pixel patterns. This analy- 
sis is performed for all binary patterns that are significant. 

In this process of template selection, significance may be 
due to attributes such as the pixel pattern's frequency of 
occurrence, the pattern's effect on the generated image, or 
both. Accordingly, if a template, i.e., a pattern of pixels in 
the binary image, is considered significant with respect to 
template inclusion in the design process, that template will 
appear in the template-matching filter. Upon operating on an 
input image, if that pattern is observed, the observed target 
pixel value will be assigned or associated with a certain 
value, i.e., a corresponding gray-scale value. Both the 
observed neighborhood and the corresponding gray-scale 
value may be stored in the look-up table. Accordingly, the 
look-up table accepts input values and outputs a desired 
corresponding output value, i.e., maps input values to an 
ideal corresponding output value. 

However, it should be apparent that this input/output 
process may be performed in various other ways without 
using a look-up table. One alternative approach that is 
equivalent to using a look-up table representation is a 
Boolean logic representation. In the Boolean logic 
representation, pixel values are used as variables in the logic 
architecture, such as a logical sum of products. The goal of 
template filter design using Boolean logic representation is 
to derive optimized Boolean operators, preferably statisti- 
cally optimized Boolean operators. 

As illustrated in FIG. 1 there is shown the basic process 
for template matching based on an observed image. Initially 
an observed image will occur (10) from which an ideal 
image is warranted (15). In order to generate an image as 
close as possible to the ideal image, a generated image (25) 
is created by utilizing a template matching operation (20). 
With the above understanding of the image processing 
setting we see that it is greatly desired to produce a gener- 
ated image as similar as possible to the ideal image. 

The conventional template matching operation is a binary 
matching process whereby a binary template or filter is to 
applied to a binary image. 

For example, as illustrated in FIG. 2A a binary image (50) 
is initially sent to a data buffering circuit (55). After which, 
a binary pattern matching operation (60) is performed. 
Binary templates are typically defined as possessing ones, 
zeros, and "don't cares." With reference to FIG. 2B there is 
shown a typical binary matching template structure. For 
example, the binary templates in this example are defined 
with ones and X's for use in the binary matching step, where 
"X" denotes "don't care". After combining the buffered 
image data and the templates in the binary pattern matching 
operation (60), an enhanced data buffering step (65) to create 
the enhanced image (70) is exercised. 

Illustrated in FIGS. 4A, 4B, and 4C is the set of images 
associated with an image restoration application using a 
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filter defined by templates representing particular Boolean With respect to a preferred statistically optimized design 

functions applied to a received image. FIG. 3 A shows a procedure, an observed image and the ideal image may be 

representation of a 3x3 window and the positions of pixels received in the form of digital images. In accordance with 

(Xj, . . . , Xp). The filter defined by this window and the conventional methods for designing a filter and the tem- 

templates of FIG. 2B possesses the Boolean function rep- 5 plates with the filter, the design method tries to map as 

resentation shown in FIG. 3B as follows: accurately as possible the observed image to an image that 

accurately corresponds to the ideal image given certain 

x 2» - • - . *9Ws +Jf 4 x o+*i *s>+*2 X » +Jf 3 *7 0) constraints such as Limited number of templates. 

When employed as a translation-invariant filter, the With reference to FIG. 1, the relationship of an observed 

singleton template x 5 behaves as an identity operator: what- 10 ideal ^age* matching templates and a generated 

ever pixels are valued one and zero in the input image are is shown. The observed image and the ideal image 

valued one and zero in the output image, respectively, ORed represent a typical "before" and "ideal after" set of images, 

onto that identity image is the result of each 2-pixel logical respectively. In other words, the observed image and the 

product. The templates corresponding to those products ldeal can be utilized 88 training images for a filter 

possess structures that straddle the origin pixel. In this 15 design procedure. The observed image and the ideal image 

configuration a template can "fit", or yield a 1, when ma y be actual mages that are selected based on expert 

positioned about a hole or break in a character stroke. knowledge, or they may contain particular structures that are 

Equation 1 is an example of an image processing operator uscful to training. For instance, they may contain precise 

that can be employed to repair breaks in character strokes patterns of particular concern to a specific image processing 

within an image. FIG. 4 is an example of applying Equation tas ^* 

1 as a filter operating on an image, where FIG. 4A shows an To further explain, in the filtering operation, given an 
ideal image, 4B shows an input image, and 4C shows the observed image, in some cases it is possible to generate an 
image resulting from applying Equation 1 as a filter to the image that is identical to the ideal image. Typically, the 
image of FIG.4B., The filter defined by Equation 1 produced generated image will not be an exact replica of the ideal 
a character (120) more likely to be recognized in character 25 image because the filter operation would require knowledge 
recognition operation. The restoration is not perfect. A more that may not be present. In addition, exact generation of the 
complicated (more and different products) filter could ideal image would most likely involve exact matching of 
achieve better restoration. As shown in FIG. 4A and FIG. 4C patterns in the gray-scale image. Such matching is imprac- 
there are still differences between the ideal image 100 and tical due to the large number of templates and processing 
the generated image 120 resulting from the use of the 30 time that would be required for most applications. This 
filtering process defined by Equation 1. situation is the result of time constraints, memory 
As illustrated in FIG. 5Aa system of the prior art is shown constraints, and other system and resource constraints. Fur- 
where an input image 150 is processed by a process 165 that ther it is recognized that the human visual system has certain 
includes binary-template-based filters 160 and combinato- limitations in what it can perceive. The excessive compu- 
rial logic 170 [changed t to match the figure]; from which an 35 tation and storage required for exact pattern matching in 
output image 175 is generated and subsequently sent to a gray-scale may yield results that are beyond the perceptive 
digital printer, high resolution printer or high resolution capabilities of the human visual system, or in general, 
display 180. beyond the limitations of the system that will receive the 

image. Given the above practical system constraints and 

SUMMARY OF THE INVENTION 40 limitations, a filtering approach using loose- gray-scale tem- 

T *jj plate matching has been proposed. In a practical setting with 

Loose-gray-scale template matching is a windowed r t . . . * * t r A . f.. 

/<si» \ *u . * 4*i • realistic limitations the loose-gray-scale template matching 

operation (filterl that operates on gray or partial gray images „ . , 6 ' . x f , , & 

* A • r „ /„ r _ ° l 0 tU Ui process allows a certain number ot templates to have a better 

and signals. In some ways it can be thought ot as an . • t * , . ^ r j . 

• c -j i j , • . i . ... match with an observed image to form a generated image 

extension of the widely practiced binary template matching 45 , . „ 5 14 ,.. 4 f 

,t j • # 1 t> -1 4 i 1 • closer to the ideal image, than would an attempt to utilize the 

methods into gray-scale. Template matching operations per- t * . . , 

r . . • j * /1 * j*u . 1 i f • same number of exact matching templates on the same 

formed on binary data (line width control, resolution t . , r 

u > j j 1 * \ u u 1 u gray-scale im age. If practical limitations with realistic limi- 

enhancement, document clean up, etc.) should also be per- f ' . . * j . . . u r 

£ , , - , 7. . j . *• v • tations do not allow for many templates, the results of exact 

formed on partial gray imaged obtained via antialiasing or , . , . * V 1, , 

scanning. Exact matching in gray-scale would often require 50 ° D ****** would be ^ 

a prohibitively large number of templates, hence loose fitting P?° r - ™* n , ew P rocess mm™ costs and generating time 

has been developed so that a range image signals could be f* 6 lde al ™ a f > and ^ ,he *"™*> f 0 " 15 on 

matched by a single template. As with binary template details mat human visual percepuon and subsequent 

matching, loose-gray-scale template matching would typi- ™>f P'°«*™g »°dules wiU actually detect. In add.uon 

cally be performed using groups of templates where each 55 the , b h ^ den * « f P^tical number ot exact 

template or subgroup of templates is associated with a mato * , tem P 1 ^ s B , tnll, f ed * desi 8 mn 8 » ™ch waller 

particular output or action. set of loose fittm 8 tem P lates - 

In addition to extending binary template matching appli- t ^ P™ eat invention separately provides the methods 

cations to gray-scale, the loose-gray-scale template match- may be used ui conjunction with known methods for 

ing method introduces an additional tool that can well «o hn6 wkMi nx******. resoluUon conversion, resolution 

address other significant image process problems such as enhancement, image restoration and the like, 

enabling a pass through mode for scanned halftones, feature The present invention separately provides systems and 

recognition, and segmentation. As with other windowed methods that may be used in conjunction with filter design 

operators, loose-gray-scale template matching can be per- and implementation techniques to optimally yield images 

formed adaptively and in several dimensions (e.g., 65 processed to better visual quality. 

2 -dimensional spatial images, 3-dimensional topological In accordance with one embodiment of the system and 

images, multiple color planes, frames in a time sequence). methods of the present invention there is provided a tem- 
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plate fitting method for fitting a plurality of templates io FIGS. 4A-C show a series of diagrams of an example of 

correspondence with a gray-scale image, comprising: an ideal image, an observed image with holes and breaks and 

receiving gray-scale image information of an observed an image resulting from a template matching operation on 

image; defining a window of said gray-scale image infor- the observed image. 

mation so as to enable a loose-gray-scale fit between said 5 piG. 5A shows a diagram of a system configuration using 

defined window and said plurality of templates; extracting a a template matching filter system according to the prior art. 

plurality of pixels of said image information using said pig. 5B shows another diagram of a system configuration 

window to which to fit at least one of the plurality of usi a fc kte matchin filtcr tcm of mc t 

templates; and fitting one of the plurality of templates to said invention 

defined window within a threshold looseness interval asso- io * _,. , ... 

ciated with the template wherein a looseness interval F * G * 6 * a ^traUng > gray-scale matching 

between the window of image data and the template is less configuration in accordance with the present invention, 

than the threshold looseness interval. FIG- 7 is a flowchart illustrating a serial loose-gray-scale 

Inaccordanc^withanotherembodimentofthesystemand matchin S s y stem * accordance with the present invention, 

methods of the present invention there is provided a multiple ™ FIG. 8 is a detailed flowchart illustrating a serial test for 

template fitting method including a plurality of single tern- loose-gray-scale template matching of a family of templates 

plates or plurality of families of templates for correspon- ^ accordance with the present invention, 

dence with a gray-scale image, including receiving image FIG. 9 is a flowchart illustrating a parallel test for a 

information of an observed image; denning a window of the loose -gray -scale template match with one template in accor- 

image information so as to enable a loose gray-scale fit 20 dance with the present invention. 

between the defined window and the plurality of singular or FIG. 10 is a diagram illustrating a parallel test for a family 
families of templates; and fitting one of the plurality of of templates performing matching operations using loose- 
singular or families of templates to the defined window gray-scale template matching in accordance with the present 
within a threshold looseness interval and outputting an item invention. 

of data indicating a result of a correspondence between a 25 FIG u ^ a di agram 0 f multiple families of templates 

single or family of templates and the received image infor- performing a template matching operation using loose-gray - 

mation within a threshold looseness interval. sca i e template matching in accordance with the present 

In accordance with another embodiment of the system and invention, 
methods of the present invention there is provided a loose- 
gray-scale template matching method including at least an 30 DESCRIPTION OF THE PREFERRED 
initial point for locating within the received image a plural- EMBODIMENT 
ity of pixel points surrounding said initial point, the method p or a general understanding of the present invention, 
including the steps of: locating said initial point and said reference is made to the drawings. In the drawings, like 
plurality of pixel points, generating a looseness interval reference numerals have been used throughout to designate 
about said located initial point with template information identical elements. In describing the present invention, the 
associated therewith, determining which one of a plurality of following term(s) have been used in the description, 
templates for fitting fits within a threshold looseness interval ^ term « bitmap » refers herein to a digital image quan- 
and, outputting an item of data indicative of whether a tizcd m a variety of formS) such as 5inary> 8 bits/pixel> or 
template is within the looseness threshold thereby indicating some intermediate number of bits/pixel, or some high reso- 
that the template is in agreement therewith. Mon quantiza tio Q state. 

Loose fit generates a template match identifier that may be ^ term « data » rcfcrs hercin to pnys i ca i s i grja ls that 

used in subsequent processing modules, such as a module indicate or include information. When an item of data can 

that would determine an enhanced output signal. indicate one of a number of possible alternatives, the item of 

Accordingly, in some embodiments the system and meth- 45 data has one of a number of "values." For example, a binary 

ods of embodiments of this invention provide a method to item of data, also referred to as a "bit," has one of two 

decrease the cost of printer electronic hardware to practical values, interchangeably referred to as "1" and "0" or "ON" 

levels for example by designing a filter possessing a rela- and "OFF* or "high" and "low." A bit is an "inverse" of 

tively small number of gray-scale templates. another bit if the two bits have different values. An N-bit 

These and other features and advantages of the system 50 item of data has one of 2^ possible values, 

and methods of this invention are described in or are The term "data" includes data existing in any physical 

apparent from the following detailed description of the form, and includes data that are transitory or are being stored 

drawings and of the preferred embodiments. or transmitted. For example, data could exist as electrical, 

electromagnetic or other transmitted signals or as signals 

BRIEF DESCRIPTION OF THE DRAWINGS 55 s{or ed in electronic, magnetic, or other form. The terms 

FIG. 1 is a diagram illustrating a conventional template j™S e video data, and pixel are interchangeably used 

matching filter paradigm. herem t0 describe terete digital signals that represent the 

t-t^ a active (on) or inactive (off) state of an element within a 

FIG. 2A shows a flowchart of a conventional binary \J Iq Qr crosshatched rtkms 

template matching arrangement. 6Q of image bitmaps hcrein arc {q reprcscnt 

FIG. 2B shows a diagram of an example of a senes of black or active pixels (having a value of 1 in a binary 

templates used in a binary matching template arrangement. representation) within the bitmaps. Such a representation is 

FIG. 3A shows a representation of a 3x3 window and the not intended to limit the present invention, but to simplify 

positions of pixels (xj x 9 ). the description thereof. Accordingly, the present invention 

FIG. 3B shows a diagram of a gate structure of the 65 may be operated in the same manner by substituting white 

template matching operation for the templates of the pixel states wherever black pixels are indicated, and vice 

example in FIG. 3A. versa. 
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"Circuitry" or a "circuit" is any physical arrangement of An item of data "defines" an image when the item of data 

matter that can respond to a first signal at one location or includes sufficient information to produce the image. For 

time by providing a second signal at another location or example, a two-dimensional array can define all or any part 

time. Circuitry "stores" a first signal when it receives the 0 f an image, with each item of data in the array providing a 

first signal at one time and, in response, provides substan- 5 value indicating the color of a respective location of the 

tially the same signal at another time. Circuitry "transfers" image. 

a first signal when it receives the first signal at a first location „ , . . . . . „ , „ . ,„ Tr 

and, in response, provides substantially the same signal at a * ac ^ cat ? n m **™ & m ^ be called a «pncL Hence, 

second location. An example of a circuit is a data or address * P 1 ^ 1 15 smallest ?*8 menl an image is 

bus in an electromechanical system such as a printing ^^cd or accessed in a given system. In an array defining 

system or computer 10 * a which each item of data provides a value, each 

A "data storage medium" or "storage medium" is a vdue indicating the color of a location may be called a 

physical medium that can store data. Examples of data *P ixel value " Each P ixel value »> for sample, a bit in a 

storage media include magnetic media such as diskettes, "binary form" of an image, a gray scale value in a "gray 

floppy disks, and tape; optical media such as laser disks and sc *l e form" of an image, or a set of color space coordinates 

CD-ROMs; and semiconductor media such as semiconduc- 15 in a "color coordinate form" of an image, the binary form, 

tor ROMs and RAMs. As used herein, "storage medium" gray scale form, and color coordinate form each being a 

covers one or more distinct units of a medium that together two-dimensional array defining an image. The invention will 

store a body of data. hereafter be described in terms of a single-color gray-scale 

"Memory circuitry" or "memory" is any circuitry that can embodiment, however, there is no intent to limit the inven- 

store data, and may include local and remote memory and 20 tion to sucn a single-color system as it has application to 

input/output devices. Examples include semiconductor multicolor systems as well. 

ROMs, RAMs, and storage medium access devices with An item of data "relates to" part of an image, such as a 

data storage media that they can access. A "memory cell" is pixel or a larger segment of the image, when the item of data 

memory circuitry that can store a single unit of data, such as has a relationship of any kind to the part of the image. For 

a bit or other n-ary digit or an analog value. 25 example, the item of data could define the part of the image, 

A "data processing system" is a physical system that as a pixel value defines a pixel; the item of data could be 

processes data. An "image processing system" is a data obtained from data defining the part of the image; the item 

processing system that processes image data. A "data pro- of data could indicate a location of the part of the image; or 

cessor" or "processor" is any component or system that can the item of data could be part of a data array such that, when 

process data, and may include one or more central process- 30 the data array is mapped onto the image, the item of data 

ing units or other processing components. maps onto the part of the image. 

An "array of data" or "data array" or "array" is a com- An operation performs "image processing" when it oper- 

bination of items of data that can be mapped into an array. ates on an item of data that relates to part of an image. 

A "two-dimensional array*' is a data array whose items of 35 Pixels are "neighbors" or "neighboring" within an image 

data can be mapped into an array having two dimensions. when there are no other pixels between them or they meet an 

An item of data "defines" an array when it includes appropriate criterion for neighboring. If, for example, the 

information sufficient to obtain or produce the array. For pixels using adjoining criteria are rectangular and appear in 

example, an item of data defining an array may include the rows and columns, each pixel may have 4 or 8 adjoining 

defined array itself, a compressed or encoded form of the 40 neighboring pixels, depending on the criterion used, 

defined array, a pointer to the defined array, a pointer to a The "looseness interval" of the present invention is a 

part of another array from which the defined array can be measure of the difference between a template and an 

obtained, or pointers to a set of smaller arrays from which observed neighborhood of pixels. Each pixel can possess a 

the defined array can be obtained. looseness interval and a template and neighborhood can 

"Control circuitry" is circuitry that provides data or other 45 possess an overall looseness interval. In the pixel- wise case, 

signals that determine how other components operate. For the looseness interval could be a simple metric, such as the 

example, "instruction circuitry" is control circuitry that absolute value of the difference between an observed pixel 

provides items of data indicating instructions to a compo- value and a template value. More complicated intervals may 

nent that includes processing circuitry. Similarly, "address- also be defined, such as intervals that are dependent upon the 

ing circuitry" is control circuitry that provides items of data 50 sign of the difference, or are a function of the difference. The 

indicating addresses to a component that includes memory function may be arithmetic, such as multiplication by a 

circuitry. factor, algebraic, such as being raised to a power, or some 

Control circuitry provides signals that "control" transfer other algorithmic means to represent a degree of difference 

of data by bus circuitry if the signals determine sources and between the observed neighborhood of pixels and the tem- 

destinations of the transfers of data by the bus circuitry. For 55 plate. An overall template looseness interval could be a 

example, the control circuitry could provide signals to a combination of the pixel looseness intervals, or it could be 

source so that it provides an item of data to the bus circuitry; a function directly applied to the observed pixel neighbor- 

the control circuitry could also provide signals to one or hood and template, where that function provides a degree of 

more destinations so that they receive the item of data from difference. Example of the overall template looseness inter- 

the bus circuitry. 60 va ^ arc averaging of at least a portion of the pixel looseness 

An "image" may include characters, words, and text as interval, and to taking the maximum of the pixel looseness 

well as other features such as graphics. A text may be intervals. In this patent specification we often use looseness 

included in a set of one or more images, such as in images interval to denote either or both a pixel looseness interval 

of the pages of a document. An image may be divided into and an overall template looseness interval, 

"segments " each of which is itself an image. A segment of 65 The "threshold looseness interval" is the maximum allow- 

an image may be of any size up to and including the whole able value for the looseness interval that indicates a loose - 

image. fitting match. As with the definition of looseness interval, 
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threshold looseness interval may be defined pixel-wise or 300. If none of the templates matched a different fit decision 

overall-template-wise. is defined, such as "do not change the value of the input 

FIG. 5b shows gray-scale image data 181 inputted to a 310 " An example of a fit decision is an address that 

digital printer 184. The printer includes a line width could be used to access an output value stored in a look-up 

enhancement circuit 182 and line width enhanced bitmap 5 table - The address could be a digital value formed of several 

183 output to a marking engine 185 to produce an enhanced binary bits. If a fit does not occur the NO decision is passed 

printed 186. Line width enhancement in such a digital back to the template storage module where another template 

printer is typically performed for one or more of several may be accessed for a subsequent matching test. When all 

possible reasons. One reason for the enhancement is com- toe templates are applied the image processing operation 

pensation for marking process characteristics. For instance, 10 cou ld return to the original image and extract a different 

a given marking process may not be able to print single pixel neighborhood of observed pixels for further matching tests, 

fines, and therefore it is desired to grow those lines in the Note tnat the matching tests could proceed until all tem- 

digital image prior to printing to enable their representation plates are examined to create a collection of fit decisions, or 

on the final printed image. Another reason for line width me matching operation could have concluded upon obtain- 

enhancement is user preference, where a given user may 15 ing a successful match. A process for a serial implementation 

prefer a particular darkness or thickness of character strokes, for matching multiple templates is shown, 

and an enhancement operation transforms the strokes from Mathematically equivalent matching processes could be 

a received width to the preferred width, employed in indirect matching operations, these would for 

FIG. 6 shows one embodiment of a generalized functional example include match identifiers which represent a window 
block diagram of a loose-gray-scale template matching filter 20 of image data of the received image and match identifiers 
generation system 250 according to this invention. The representing the templates and associated threshold loose- 
template matching system 220 is connected to a template set ness intervals. 

and receives as input the gray -scale image 200 over a data FIG. 8 further teaches methods with a serial implemen- 

buffer 210. The gray -scale image can also be received over tation of loose gray-scale template matching. With reference 

a line signal or other link. The gray-scale image buffer 25 to FIG. 8, in another embodiment of the present invention a 

provides a variety of image data to the loose-gray-scale serial loose-gray-scale template matching system is shown 

template matching system 220. where first a target pixel is initialized 400. About the target 

In general the data source can be any one of a number of pixel, a set of template values is read in 410 and a particular 

different data sources as an example such a source can be a 3Q re g ion of support for a current template is determined, 

scanner, digital copier, digital camera or any known device Another way of looking at this concept is a set of templates 

suitable for electronic source generation and/or storing or ^ neld m memory, and successive windowed neighborhoods 

transmitting the electronic image. Further, the data source arc compared to the stored templates. From the region of 

can be multiple devices hooked up serially or in parallel with su PP OTt extraction of the group of coincidental pixel 

the template matching system. val ™s from mc image relative to the target pixel is per- 

The data received from the data source is input to the formed <** for a loose-gray-scale template fit is 

loose-gray-scale template matching module, where one or Performed 440 and if the result 1S NO then the sequence 

more sets of tem lates are applied to the image The signal proceeds to the next template 450 until all the templates are 

oumllm theTemplate matching block could be a match * sted 460 U * e te f <* * c l°°se-gray-scale ma <ch * YES 

identifier, which could be passed onto a module for final 40 then an action based on loose-gray-scale match is generated 

signal selection, or the matching module itself could gen- tor me tar § et ?™ e] 47U * 

erate the final output signal by outputting the output signal In reference to FIG. 9, there is shown an embodiment of 
associated with a matched template. The output signal could a parallel test 800 functionally denoted by f for a single 
be a variety of forms such as a gray-scale pixel value, a template T performing a matching operation using loose- 
binary pixel value, a group of binary of gray-scale pixel 45 gray-scale template matching according to the present 
values, or an item of data describing the match condition. invention. In particular, the present invention receives a 
With reference to FIG. 7, a process for a serial imple- series of input pixels (x>, x^ . . x„) 810 where N is the 
mentation for matching multiple templates is shown. lUe f ize of me neighborhood in number of pixels and applies a 
loose-gray-scale template matching system includes an loose-gray-scale match 

input image which is the output of a data source and a set of 50 Starting with the definition of loose-gray-scale template 

templates 300. The input image 310 is subsequently con- matching, several implementation and processing schemes 

nected to locate target pixel circuit 320 for determining a us mg sets or multiple sets of templates will be described 

target pixel. Surrounding the target pixel 330, a window is below. 

designated so as to extract a defined portion of the image For a single template T there is a loose match between the 

about the target pixel 330. Such a window could be a 3x3, 55 digital image in the observation window and the template 

4x4, 7x4 . . . , etc. type of matrix. Loose-gray-scale when each pixel of the observation window, has an absolute 

templates are stored in a template storage module 300. A value difference 820 between the image value at each pixel 

template from the template storage module is input to the x ; and the template value for each pixel t,- that is less or equal 

template comparison module 340. A loose-gray-scale tem- 830 to the chosen looseness interval threshold 6^. The 

plate match is performed 340 on the defined portion of the 60 aforesaid absolute value difference in this embodiment is the 

image data. If a loose match occurs a fit decision is created. looseness interval for each pixel in the window, and the 

A fit decision could be a code that identifies the matching limiting value for looseness acceptability for a match at each 

template, or it could be some resultant value, such as an pixel is the looseness interval threshold. A loose-gray-scale 

output value or statistic of the image. If a loose match does template is composed of the template pixels values t ( - and the 

not occur with the given template then the next template 65 threshold intervals b if or a single threshold interval for the 

from the set of templates 300 is tested for a loose match, template In this embodiment we employ looseness intervals 

until there are no templates 350 to loosely match in the set and looseness interval thresholds for each pixel in the 
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window, aad all, or some predefined number of looseness denoted as h. Template matching niters will usually be 

intervals must be less than the threshold looseness interval composed of multitude of template, which refer to as the 

to define a loose match. The individual looseness interval template set or a template family. Here, the values for each 

decisions are evaluated in concert in combine block 840 to template are stored into memory as well as the correspond - 

generate an overall fit decision 850. 5 *ng threshold looseness intervals In the figure the template 

The template fitting process may be formulated as shown v f es ar * channeled simultaneously to the processing step 

■i^ ^ l •* • l ijc i where a looseness fit is ascertained for each template m 

in Eqs. 2 and 3 where it is chosen to define a loose fit as lleL Afit dcdsion is ted for eacfa loose ma P tch test 

having all pixel looseness intervals possess values less than ^ ^ ^ information \ combined in a manner to reach 

or equal to their respective looseness interval thresholds, ^ ^ ovcrall fit dcdsion of whcthcr a particular (emplatc 

ff - - . . Ir loosely matched or not. By performing a parallel loose 

'n . VJ-miiHV^ match operation more decision making steps can be per- 
formed in a shorter interval of time. Parallel implementation 

where is a preferred embodiment in applications implemented in 

hardware, such as an Application Specific Integrated Circuit 

Wi^O" 1 tf o$u£b { and 0 otherwise (3) 15 (ASIC), whereas serial implementations as shown in FIG. 7 

are typically preferred for software implementations, 
and where I is called the characteristic function or indicator in a template-wise parallel implementation, for a collec- 
function and |I is the absolute value. tion of M templates, we can say that there M loose matching 
We have expressed a loose-fitting template as a set of template operators f(*, T,-) i =1, . M, operating in parallel 
pixel values and corresponding threshold looseness inter- 20 for a set of templates (J l9 T 2 , . . . T^). The results of each 
vals. There are alternative representations, such as two individual template match decision are combined in a man- 
templates, one that specifies the lower bound that pixels may ner so as to generate the overall template set fit decision, 
possess and the other template specifies the upper bound that FIG. 10 illustrates an embodiment of this template-parallel 
pixels may possess to qualify as a loose-fitting match. That method where a serial input of digital image data is con- 
representation can be written as the following equation, 25 verted in parallel and combined to make the overall template 

fit decision. The combine block 740 can take a variety of 

A x i> X wl)-A x i> x 2> • ■ • » x v 'i> f 2> * ■ ■ > V) ~ min U[«/-6»] forms. If the template fit decisions are disjoint, that is, only 

(*)» • ■ • > (4) one t em pi a te can loosely fit, the combine block can simply 

WQ ere select the match decision that yielded a match. If the match 

30 decisions are not disjoint, the combine block must in some 

l [aJ . m (U)**i if a,<=«<»b / and o otheiwise (5) cases arbitrate between multiple matches. In this case we 

have found it beneficial to give priority to certain templates 

One advantage of this representation is that the threshold l0 allow the combine block to select a match of higher 

looseness interval is not limited to be symmetric about a priority. Useful priority criteria have been based on template 

central value. 35 s i ze (number of pixels) and threshold looseness intervals. A 

There are other mathematical means to express the match- smaller threshold looseness interval would tend to indicate 

ing condition as described in Eqs. 2 and 3. One such a m0 re exact match and would be given priority in typical 

equivalent form that we have found useful is provided in Eq. applications. Note that other related arbitration criteria fall 

6, within the scope of the present invention. 

40 Also, various templates may be grouped in sets wherein 

■ X "Tm ** ' ' ' ' ^ ' ' " & each set seives a P articuJar Processing task. Within each 

group, the templates may be arranged or a general arbitration 

Where procedure may be defined. The arbitration procedure may be 

a logical expression or a functional combination of the 

■ s ' =r ' +6 ' 45 individual loose matching results. A family-parallel embodi- 

d ^_ a ^ ment of this concept is shown in FIG. 11. In FIG. 11 the pixel 

values of the observed neighborhood are inputted to multiple 

Hence, there are a multitude of means to compute loose parallel matching channels, where each will apply a filter 

matches according to the loose-fitting criteria of the present defined by a family, or set, where matching with a set of 

invention. Another example is that loose matching could be 50 templates is shown in FIG. 10. 

determined by performing exact matching operations at It should be noted that templates sets and threshold 

lowered quantization, that is, an exact match using only the looseness intervals could be modified during the processing 

upper three bits of an eight bit pixel value is a loose match of an image. For instance, tag information indicating image 

with respect to the pixel value at 8 bits. It is the intention of type could select a particular set of templates to be applied 

this patent to cover all such equivalent forms of the loose- 55 to that image type, or select a threshold looseness interval 

fitting criteria. that is suitable for that image type. Further, prior processing 

Note that alternative looseness criteria may be applied. history could be incorporated into loose-gray-scale template 

For instance, an overall template looseness interval may be matching. For instance, although it has been described to 

computed by processing the individual pixel looseness inter- utilize window pixel values solely from the observed image, 

vals with an operation such as averaging a selected group, or 60 recursive methods could be employed where the window of 

taking their collective maximum value. That overall tern- pixels applied to the matching operations could contain 

plate looseness interval could then be compared to an overall some mix of observed pixels values and recently processed 

threshold looseness interval to establish a match or no match pixel values. The above discussion simplifies the teaching of 

condition. the general method by referring to either of these windowing 

In FIG. 10, there is illustrated another embodiment of the 65 methods as extracting observed pixels, 

present invention of a parallel test for loose-gray-scale Further, pending U.S. patent application Ser. No. 09/222, 

template matching for a set of templates functionally 919, filed Dec. 30, 1998, describes a method of "vector 
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template matching" where some observed values may be values, a lower value and upper value associated with the 

extracted from other dimensions. An example is the extrac- threshold interval, or a reduced quantization state with 

tion of pixels values from multiple frames in a time respect to a quantization state of the received image. The 

sequence, multiple dimensions in a color space method of claim 2 further said matching step using template - 

representation, and multiple dimensions in space. In the 5 wise looseness intervals instead of said pixel-wise looseness 

present invention, observed pixels may be extracted from interval values. 

such multi-dimensional sources. 5 method of claim 2 , wherein processing circuitry 

Although the present invention has been described and mcmdcs ^^mfc operations for associating the templates 

illustrated in detail above and through the Figures, it is ^ the gray ^ cale image Informa tion. 

intended that the spirit and scope of the mvention be 10 fi ^ ^ fi(tm method of ^ j ^ 

^^^fk aS 1* ?! s ^l 01 ^ 0 ^^ °i ^ ™ » sponding candidate templates either in a parallel or serial 

and that the same be limited only by the appended claims as r & r r 

interpreted in light of the foregoing and all other equivalents. ^ > ™ Ce f?' . , e , - + . ^. 

What is claimed is* 7. The method of claim 1 further outputting gray-scale, 

1. A method for fitting a plurality of candidate templates is hi S h addressability, image statistics or templates match 

in correspondence with a gray-scale image, comprising: identifiers generated by said determining step. 

, . ■ e t . c u j 8. The method of claim 1. wherein said extracted gray- 

receivmg gray-scale image information of an observed , . . c . * . c 

. scale image mformation comprises pixel tile information or 

, „ . . , , . , , , .,,, JL a predetermined matrix size. 

defining a window which bounds a neighborhood about a 9 ^ method of daim 8 whcrcin ^ outputtcd enhanced 

target area within said image; dk mforraatiorA 5eing high addressability information. 

extracting gray-scale image information relating to said io. The method of claim 8 wherein said matching step 

bounded neighborhood; generates a match identifier to match said pixel tile infor- 

extracting gray-scale information from said candidate mation. 

templates; 25 The method of claim 8, wherein pixel-wise looseness 

for all candidate templates, determining at least one interval values are generated. 

looseness interval value by an application of a pre- ^ The method of claim 11 further combining said 

defined looseness interval function f(x,t) where x is pixel-wise looseness values to form template -wise looseness 

said extracted gray-scale image information and t is values. 

said extracted gray-scale information from said candi- 30 *3 The method of claim 11 further storing template pixel 

date template information; ^le information and said pixel-wise looseness values. 

. c . . , , . , , x a . 14. The method of claim 13 wherein said templates and 

defimng at least one looseness threshold 5 for each ..... t . , , , . A , , A . 

i j • j i • * t 1 said pixel-wise threshold looseness intervals are stored in a 

determined looseness interval value; r . . - . - . c 

representation which enables fitting one of a plurality of 

comparing each of said looseness interval values to its sin k Qr famiUes of templatcs to said pixel Ulc in f orma tioD. 

respective looseness threshold 8 such that templates 15 ^ method of daim u comprismg . 

having a looseness value less than or equal to its . . .. A . t . . A . 1A . . . 

respective looseness threshold 8 are identified as wherein ^ representation being either multiple nominal 

loosely fitting templates; f lxel ^ ™ th incremental corresponding upper and 

- .^.1^. , lower threshold values, a lower value and upper value 

designating any of the identified loosely fitting templates 4Q associated with the threshold interval, a reduced quan- 

as a match; and tization state with respect to a quantization state of the 

fitting at least one of any of said Identified matching received image. 

templates to said defined window of image data. 16. The method of claim 11 further using arithmetic or 

2. The method of claim 1 further generating using pro- algorithmic processes to calculate the pixel- wise looseness 
cessing circuitry a looseness interval between window of 45 interval values. 

received image and each template associated therewith; and 17. The method of claim 8 further said matching step 

comparing the threshold looseness interval to the loose- using template-wise looseness intervals instead of said 

ness Interval using the processing circuitry, pixel-wise looseness interval values. 

3. The method of claim 2, wherein said templates and said 18. The method of claim 17 further using arithmetic or 
threshold looseness intervals are stored in a representation 50 algorithmic processes to calculate the template-wise loose- 
which enables fitting one of a plurality of singular or ness values. 

families of templates to said defined window. 19. The method of claim 17 further storing said templates 

4. The method of claim 3, wherein said representation and template-wise looseness values, 
being either one or more of: multiple nominal pixel values 

with incremental corresponding upper and lower threshold * * * * * 
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